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The  Renewal  Equation  for  Markov  Renewal 
Processes  with  Applications  to  Storage  Models 

by 

Eric  S.  Tollar 


Abstract 


For  Markov  renewal  processes  in  which  the  sojourn  times  are  controlled 
by  an  imbedded,  denumerable  state  Markov  chain,  it  is  shown  that  there  exists  • 
a  random  time  at  which  the  Markov  renewal  process  regenerates.  The  basic 
renewal  theorem  is  then  applied  to  determine  the  limiting  behavior  of  the 
Markov  renewal  process .  These  results  are  applied  to  a  particular  two  com¬ 
partment  storage  model  to  determine  the  limiting  behavior  of  the  amounts  in 


The  Renewal  Equation  for  Markov  Renewal 
Processes  with  Applications  to  Storage  Models 


I.  INTRODUCTION 

Let  J  be  a  denumerable  set,  and  let  {X  ,n=0,l,...}  be  a  stationary, 

n 

positive  recurrent,  aperiodic,  irreducible  Markov  chain  with  state  space 

J.  Let  jr  be  the  stationary  measure  of  (Xn>.  For  an  arbitrary  space 

(S,F),  let  {Zn,n=0,l,2, . . . }  be  a  process  defined  on  (S,F)  such  that 

{(X  ,Z  ),n=0,l,2, . . . }  is  also  a  stationary  Markov  chain  with  transition 
n  n 

probabilities 


Pn(i,y: ( j ,A))  -  P(Xn=j ,Zn€A|X0=i,Z0=y), 


(1.1. 


for  i,  j  e  J,  y  e  S,  A  €  F . 

Let  0  = Tq  <  T^  £  £  • • •  be  a  sequence  of  random  variables  defined 

such  that { ( ( X  ,Z  ),T  ),n=0,l,...}  is  a  Markov  renewal  process,  where 
n  n  n 

for  t  5  0,  A  e  F, 


P(T  St,ZneA|Xri  .  ,X  ) 
n  n  n-l  n 

=P(T  St\Xn  . ,X  )P(Z  €A|X_  .,X). 
n  n-l  n  n  n-l  n 


(1.23 


That  is,  the  sequences  {T  }  and  (Z  }  are  conditionally  independent 

n  n 

given  {X^}. 

The  first  moment  of  the  sojourn  time  in  state  (i,z)  is  independent  of 
z,  and  given  by 


The  average  sojourn  time  we  define  by 


8  = 


l 

ieJ 


it  ,Tn .  • 
l  i 


(1.3) 


Finally,  we  define 


(X(t),Z(t)) =  (XN(t)*ZN(t) 

where 

N(t)  =  sup{n:  T^t}.  (1.4) 

There  has  been  a  substantial  body  of  work  on  semi-Markov  processes 
on  arbitrary  state  spaces,  in  general  directed  at  the  asymptotic  behavior 
of  the  process,  and  this  paper  is  no  exception.  Typically,  the  authors 
attempt  to  establish  conditions  sufficient  to  guarantee  that  the  basic 
renewal  theorem  can  be  applied  to  the  process.  The  approaches  have  been 
varied  (see  (Jinlar  (1969),  Athreya,  McDonald  and  Ney  (1978a,b),  Athreya 
and  Ney  (1978),  Kesten  (1974),  and  Nummelin  (1978)),  but  in  general  seem 
to  be  directed  at  the  creation  of  a  stopping  time,  independent  of  the 
future  process.  Athreya,  McDonald  and  Ney  used  the  properties  of  so- 
called  C-sets  of  <p- irreducible  Markov  chains  (see  Orey  (1971))  to  propose 
a  method  for  the  creation  of  an  artificial  renewal  point  of  the  process. 
Unfortunately,  the  method  does  not  generalize -to  all  Markov  renewal  pro¬ 
cesses.  Hov.'ever,  we  will  establish  in  section  2  that  for  a  process  as 


defined  above,  a  renewal  point  can  be  created.  Therefore  a  renewal  equation 
is  available,  and  results  follow  from  application  of  the  basic  renewal 
theorem . 


ft 


'•Ta 


In  the  subsequent  section,  these  results  are  applied  to  a  storage 
model.  In  a  simpler  form,  the  model  was  first  proposed  as  a  single 
compartment  model  by  Senturia  and  Puri  (1973),  with  subsequent  research 
by  Senturia  and  Puri  (1974),  Puri  and  Senturia  (1975),  Puri  (1978), 
Balagopal  (1979),  Puri  and  Woolford  (1981),  and  Puri  and  Tollar  (1985). 

The  model  was  extended  to  an  arbitrary  compartment  model  defined  on  a 
Markov  chain  by  Tollar  (1985a, b)  and  was  considered  with  two  compartments 
when  defined  on  a  semi-Markov  process  by  Tollar  (1986).  However,  in  the 
last  cited  paper,  the  case  where  both  compartments  were  subcritical  was 
left  as  an  open  question.  Using  the  results  of  section  2,  the  asymptotic 
behavior  of  the  storage  model  when  both  compartments  are  subcritical  is 
determined  via  the  basic  renewal  theorem. 

2.  RENEWAL  EQUATIONS  FOR  THE  SEMI -MARKOV  PROCESS 

While  the  structure  of  {X  ,Z  }  is  crucial  in  this  paper,  for  ease 

n  n 

of  development,  let  us  temporarily  discuss  Markov  chains  on  arbitrary 

spaces.  Let  {Y  ,n=0,l,,..}  be  a  Markov  chain  which  takes  values  on  some 
n 

arbitrary  state  space  (S,F),  with  P(v,*)  a  regular  version  of  the  station¬ 
ary  transition  probabilities.  Then  for  zeS,  A  e  F,  we  define  the  n-step 
transition  probabilities  recursively  by 


Pn(z,A)  = /P(z,dy)Pn'1(y,A). 
s 


(2.1) 


$8$ 

SKBSiS 

Let  if  be  a  non- trivial  o-finite  measure  on  (S,F). 

Definition  1.  {Y^}  is  ^-irreducible  if,  whenever  $(A)  >  0  for  AeF, 

03 

then  J  2  nPn(y , A )  >  0 ,  for  all  y  e  S . 
n=l 

Definition  2.  A  a-finite,  non-trivial  measure  y  on  F  is  called  sub¬ 
invariant  for  {Y  }  if  y(A) > /y(dy  )P(y,A)  for  all  AeF,  and  called  in- 

s 

variant  if  equality  holds. 

Definition  3.  If  there  is  a  finite  invariant  measure  y  on  F  with  y(S)  =  l, 
we  call  {Y^}  ergodic,  and  y  the  stationary  measure  of  {Y^}. 

Let  {(Y  ,T  )}  be  a  semi -Markov  process  defined  on  the  state  space 
n  n 

(S,F),  where  for  all  yeS,  AeF,  t>0,  H  ,(t)  is  a  regular  version  of 

y  h 

the  transition  function  with  respect  to  <f.  That  is. 


V(t)*P(VA,VTn-l£tlVl=y)’  <2-2) 

(for  details,  see  Cinlar  (1969)). 

The  concept  of  a  splitting  technique  using  C-sets  to  establish 
asymptotic  convergence  of  the  semi-Markov  process  has  been  proposed  by 
Hummelin  (1978),  Athreya,  McDonald  and  Ney  (1978a,b),  and  Athreya  and 
Hey  (1978).  The  pertinent  results  we  summarize  below. 


HYPOTHESIS .  There  exists  an  AeF,  an  integer  k  >  0 ,  a  probability  measure 
tp  on  S  n  A,  a  family  of  probability  measures  v(x,0)  on  1R+  for  all  x  e  A, 
and  a  constant  X,  0  <  X  < 1,  such  that  for  all  x  e  A,  E  e  F  and  D  e  B  +, 


*1 


f 


: 

*1 


f 
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a 


Si 


y 

I 


1 

<«*! 


P(YkeE,TkeD|Y0=x)SAiKE)v(x,D). 


If  this  hypothesis  is  valid,  then  Athreya  and  Ney  (1978)  establish  the 


following  result: 


LEMMA  2.1.  Subject  to  the  hypothesis,  there  exists  a  semi-Markov  process 


{(Y*,T*),  n=l,2,...}  distributed  as  { ( Y  ,T  ),  n=0,l,2, . . . },  and  a  random 
n  n  -  n  n  - 


time  N  such  that  for  all  B  e  F,  CeIB+,  zeS, 


P(Y*eB,T*eC,N<o»|Y*=z) 


=  MB)P(T*€C,N<»|Y*=z). 


For  any  probability  measure  v(*),  any  random  variable  X,  and  anyjset 


A  e  F,  we  define 


Ev(X)  =/  P(X|Y0=z)dv(z), 
s 


(2.3) 


Pv(X5A)=Ev<Ia<X», 


(2.4) 


From  lemma  2.1  it  can  be  easily  seen  that 


P^(Y(t)eB)  =  P  (Y*(t)cB,T*>t) 


+  /  P  (Y(t-x)eB)dP  (T*<x), 


(2.5) 


which  implies  that  renewal  theory  can  be  applied.  While  it  is  not  clear 
in  general  whether  or  not  the  hypothesis  is  valid  for  all  ^“irreducible 
Harkov  renewal  processes,  we  will  establish  that  it  follows  for 


Z  = {(X  ,Y  )}  as  proposed  in  section  1.  The  following  lemma  is  a  modi- 
n  n  n 

fication  of  the  proof  of  the  existence  of  C-sets  for  ^-irreducible  Markov 
chains  (see  Orey  (1971)). 

LEMMA  2.2.  Let  {X^Z^}  be  a  irreducible  Markov  chain  as  defined  in 
Section  1.  Then  for  any  set  E  c  S  and  any  j  e  J  where  $( jQ,E) >  0,  there 
is  a  k  >  0,  a_  p  >  0,  a  sequence  j^»  j2»  •••*  3^  ^  c  J»  and  a  set  A  c  E  with 
( j  0 ,  A)  >0  such  that  for  all  z  e  A,  and  all  B  c  S, 

p(Xn= jo»Xk_i=jk_i»  *  •  •  >X1=3i»Zk£B|X0=:],0’Z0=z)  *P*^o*BnA). 

PROOF.  For  convenience  of  notation,  let  <frj(C)  =  <K  j  ,C)  for  all  CcS. 
Further,  for  any  set  U  c  S  * S  let 


i)  (^(x)  =  (y:(x,y) e  U}, 
ii)  U2(y) = (x:((x,y) e  U). 


(2.6) 


Let  i,  stand  for  a  general  sequence  i. ,  i0, 
l,m  i.  t 

sequence  is  necessary,  it  will  be  specified). 
Radon-Nikodym  derivative  of 


. ...  i  ,  (if  a  particular 
m-1 

Let  p.  (x,y)  be  the 
1l,m 


P(Zm€(- )  ix0=j0,z0=x,x1=i1, . . .  ’VrVl’V V 


6 


with  respect  to  <j> .  .  Finally,  for  our  set  E,  let 
30 


=  ((x,y)  e  E  x  E:p.  (x,y)>l/n}» 
l,m  l,rn 


(2.7 


and 


CD  OO  (n  ) 

H=  u  u  u  H.  (2.8 

m=l  n=l  {i  : i .eJjl^jiir.-l}  1l,m 

l,m  3 

Clearly,  by  the  4>~irreducibility  of  {(X  ,Z  )},  we  have  for  all  xeE  that 

n  n 

<{>.  (H  (x))>0,  for  H  (x)  as  in  2.6i.  Then,  by  Fubini’s  theorem,  we  have 
JQ  x  x 


/$■  (H  (x))$.  (dx)  =  /<?> .  (H  (y))4>.  (dy),  (2.9 

30  x0  ■10 


so  <f>.  ({y:<f>,  (H  (y) ) > 0) ) > 0.  This  implies  that  there  must  be  an 
-'0  1 

(n  )  (n  ) 

n  ,  i  and  an  n„,  i.  where  for  F  =  H.  and  G  =  H.  , 

1  2  1’mo  _  i,  _ 

-L  2  l,m^  1,1^2 


<J>  •  ( <y  =  4» .  (F  (y))>0,4>.  ( (G  (y) )>0})  >  0.  (2.1( 

30  30  -'o 


Consider  finite  partions  {E^n^}  of  E,  becoming  finer  as  n  increases. 

Let  E'n]  =  E^xE[n\  Clearly  (E^11]}  is  a  finite  partition  of  E^  =  E*E. 
ct»e  a  8  ct,D 

Let  i(n,x)  be  the  unique  index  for  the  element  of  the  partition  {E^n^} 

a 

where  xeE^?^  ..  By  a  differentiation  theorem  (see  Doob  (1953)),  we 


2 

have  that  for  each  measurable  set  BcE  , 


,(n) 


i~1.2 


,(n) 


[<J>T  (E.y  .  .)]  <f.t  (BnE.‘w  .  .  )-^In(x,y), 

i(n,x),i(n,y)  '  i(n,x),i(n,y)  Bv 


(2.1 


« 


M* 


for  all  x,  yeE2-N,  where  ^  (N)  =  0  and  (j>?  =  <J> .  *$.  . 


30  30  30 


Therefore,  for  F  and  G  as  in  (2.10),  there  is  an  x0,  yQ  and  zQ  with 


x0eF2(y0)-N2(y0)’  and  z0  e  Gl^y0^  ~  Ni^yo^*  wh*ch  satisfy  (2.11).  Let 
a=i(m,Xg),  6  =  i(m,yQ)  and  Y  =  i(m,Zg).  Then  there  is  an  N  where  for  all 
m  >  N, 


♦?  (FnE<mh>(3/4H.  (E<m))£  (E<m)), 

3o  0,0  3o  “  3o  6 


(2.12 


♦?  (GnE^)S(3/4H,  (E<m))$.  (E(m)), 

J0  P,T  ->0  p  3o  Y 


For  n  >  N,  let 


A={xeE<n):$  (E^n)nF  (x))  >  (3/4)<J> .  (E^n) )}, 
30  *  L  30  &  . 


(2.13 


B  =  {z€E^n):<p.  (E^n)nG  (z))  S  (  3/4  )<^  (E<n))}. 


r  rj0  B  2 


V  6 


Clearly  4>.  (A)  >  0  and  <£ .  (B)  >  0  (otherwise  (2.12)  would  be  violated),  and 


for  xcA,  zeB,  we  have 


♦  4  (F  (x)nG  (z))  ><*.  .  (E^n ^  )/2 . 
J0  30  0 


(2.14; 


Therefore,  for  y  e  F^x)  n  G2(z) ,  (x,y)  e  F  and  (y,z)  e  G.  The  definition  of 
the  Radon-Hikodym  derivative  yields 


K  i  (x,z)s/p.  (x,y)p.  (y,z)$.  (dy) 

1)01^  l,m2  S  l,m^  1.^2  ^o 


>  f  p.  (x,y)p.  (y,zH.  (dy)  (2.15) 
F^xJnG^z)  1,1^  lsTn2 


^<KE^n))/2n1n2. 


„  Since  {(X  ,Z  )}  is  <t- irreducible ,  there  exists  an  m  >  0  and  a  c>0  where 
n  n 

<J> .  ( C  )  >  0  for 
J0  m,c 


C»,c  3  {xe  BiP«  W  £  <VA>I<  W  3  <  Vx))  >  cK  (2-16) 


If  we  consider  only  those  paths  i,  where 

l,i”. 

P(Xm=VXm-l=Vl**'*Xl=il,X0=V  >  °*  t^ien  there  must  be  a  particular 
i,  with  <t> .  (C*)>0,  where 

l’m  ^0 


C*  =  (x  e  B:P.  '  (x,A) >  c), 
1l,m 


(2.17) 


PL  <x>A) = p(  va,v  v  v^vv-  * •  •  w ' 

1  ,TTi 


Therefore,  from  (2.15)  we  have  for  xeC*,  y  e  C*  that 


p  .  .  .  (x,y)  ;>  /  (x,dz)p.  .  (z,y) 


l,m  1  ,m^  l,rn2 


A  l,m 


l»m  l,u*2 


/  ?\m)  (x,dzH.  (E<n>/2n.n  >c<|>  (E*n))/2n  n. 

A  *1,1*  •  ^0  0  12  j0  3  1  ‘ 


W vs */  *  Vv V’**  v  ‘  ,*  , 


Thus ,  for  k  =  m  +  +  m  ,  for  D  c  S,  and  for 


X  €  C*, 


it  follows  that  for  all 


P(  VD|  VVZ0=X,Xr3’l’‘  ',XK-l=jk-l,Xk=V 


>c6.  (DnC*)<J> .  (Eg  )/2n  n„. 
30  30  B  12 


From  (2.18)  the  lemma  follows,  where  A  =  C* ,  and 
P=C0  (E^n))P(Xk=j0,...,X1=j1|X0=j0)/2n1n2.  □ 


COROLLARY.  If  {(X  ,Z  )}  is  a  4> -irreducible  Markov  chain  as  in  lemma  2.2 


and  if  {T  }  and  {Z  }  are  conditionally  independent  given  (X  },  then  for 


E »  { j  ^ ) ,  k ,  A ,  and  p  as  in  lemma  2.2,  for  x  e  A ,  Cel',  DcS, 


P(\e30’ZkeB,TkeC*X0=30’Z0=x)  ~  p«j>(j0»BnA)v(c), 


where  v(f0,x3) = P(Tk<x|X0=jQ,X1=j1,. . . »Xk=j0). 


PROOF.  Certainly,  for  all  xeS, 
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P(VVVB-VC|VVV*) 


aP(X1=j1,...Xk=j0IX0=j0)P(Zk€B,Tk€C|XQ=j0,Z0=x,X1=i1,...Xk=j0) 

=  PCx^jj,. . . >  VV  VVP<  VB  ^VV'-VV-  •  •  W  <5'19) 

*  P(T]<6C|X0=j0,X1=j1,...Xk=j0), 

where  the  equality  follows  from  the  conditional  independence  of  T  and 

K 

Zk  given  {Xi:i  =  0,  1,  . ..k}. 

Therefore,  for  xe  A, 

p(VVVB>Vc,xo=VVx) 

2:  P(X1=j1,. . .  ,Xk=j0iX1=j0)P(ZkeB|X0=j0,Z0=x,X1=j1,. . .  .\sj0)v(C)  (2.20) 

apv(C)<t>(j0,BnA).  D 

From  the  application  of  lemma  2.1,  it  can  be  shown  that  a  renewal 

equation  can  be  created  for  the  Markov  renewal  process  (X  ,  Z  },  as  in 

n  n 

expression  (2.5).  Once  the  behavior  of  E,(T*)  is  determined,  the  basic 

N 

renewal  theorem  can  be  applied. 

Let  us  temporarily  let  Y  = {X  ,  Z  }.  To  create  the  renewal  point, 

n  n  n 

we  will  make  an  inconsequential  change  in  the  definition  of  the  process 
{Y*}  as  proposed  by  Athreya,  McDonald  and  Ney  (1978a, b). 


11 


For  k  as  in  lemma  2.2,  we  note  that  {(Y^,!^),  n  =  0,  1,  2,  ...} 
is  also  a  Markov  renewal  process.  From  this  process,  we  define  another 
Markov  renewal  process  { ( 6^  i  )  )  by 


P(6  =6,Y*  eB,T*  -T*  <t|6  ,  ,  =Y,Y*  =y) 

nk  nk  nk  nk-k  nk-k  nk-k  J 


(2.21) 


I(5.0)P(YiikeB.Tnk-Tnk.k£tlVnk.k=y)  if  ye  <j0,A)‘ 


p*(Bn(i0,A))v(CO,t])  t  KS.OfPiY^B.T^-T^Stly^.^y) 


-  2p$(Bn( jQ,A))v([0,t])]  if  ye  (jQ,A), 


with  p,  v  as  in  corollary  2.2,  and  y  and  6  are  either  0  or  1.  We  can 
then  define  through  the  process  {(6nk,Y*k,T*^)}  hy 


P(  n  (6  .=6 .  ,Y*  .eA,,T*  .-T*  .  ,<:t.)|Y*  ,  =y  ,Y*  =y  ,T*  -T*.  = 

nk-]  ]  nk-j  j’  nk-j  nk-]-l  j  nk-k  J0  nk  J1  nk  nk-k 


(2.22) 


=  (  n  I(6.=0))P(  n  (Y  .  .eA.,T  ,  ,-T  .  .  <t.)|Y  .  =y.,T  =y, ,T  ,  -T  ,  =t). 

\.=1  ]  nk-]  ]’  nk-j  nk-j-1  j  nk-k  J0’  nk  Jl’  nk  nk-k 

While  the  process  (6  ,Y*,T*}  is  not  a  Markov  renewal  process,  from  (2.21) 

d 

and  (2.22)  it  is  clear  that  {(Y*,T*)} =  {(Y  ,T  )}.  Also,  letting 

n  n  n  n 


N  =  inf(n  >1:6  =  1}  , 
n 


(2.23) 


it  follows  from  (2.21)  and  (2.22)  that  for 


V.V.V.V.  AV. 


♦(B) = [*< j0,A)]“1^(Bn(j0,A)), 

t 

P,(Y*(t)eC)  =  P,(Y*(t)eC,T*>t)  +  /  P.(Y*(t-T)cA)dP.(T*<T),  (2.24) 

V  N  q  V  V  N 

and  the  renewal  equation  is  satisfied. 

THEOREM  2.3.  If_  0  <»,  and  { Y^}  is  ergodic  then  for  <j>-almost  all  y  and 
any  set  C, 

CO 

lim  P  (Y(t)cC) = [E  (T*)]-1/  P  (Y*(t)eC,T*>t)dt, 
t-H»  y  v  n  0  v  h 

where  E , ( T* ) <  « . 

-  ♦  IT 

PROOF.  First  we  note  that  since  the  distribution  of  {T^}  depends  only 

on  (X  },  if  T*  is  arithmetic,  then  our  Markov  renewal  process  is  equiv- 
n  N 

alent  to  an  appropriate  Markov  chain.  As  such  we  will  assume  T*  is  non- 

arithmetic.  From  (2.22)  it  is  clear  that  E,(T*)  can  be  determined  from 

N 

{6  ,  ,  Y*  ,  T*  }..  Let  y  be  the  stationary  measure  of  (Y  }.  Define  y' 
nk  nk  nk  n 

for  Be  F  by 

y'  (1,B)  =  py(j0,A)4>(Bn(j0,A)), 
y* (0,B)  =  y(B)  -  py( jQ,A)^(Bn( jQ,A)). 


Let  us  show  that  p  'is  the  stationary  measure  of  {<5  ,  ,Y*  }.  It  can 

nK  nK 

be  trivially  verified  from  the  fact  that  for  ye  (jQ,A),  Be  F 
k 

P  (y,B) £ p$(Bn( j^.A))  that  p1  is  indeed  a  measure.  Therefore,  we  need 
only  check  definition  3  for  stationarity.  For  p'(l,B) 


I  JV  ( j ,dx)P(fi  *l,Y*eB 16  =j ,Y*=x) 
j=0  S 


=  /C  I  V*  ( j,dx)3P(6  =l,Y*eE|Y*=x) 
S  j=0 


=  /v(dx)P(6k=l,X*eB|Y*=x)  =  p(j0,A)p^(Bn(j0,A)), 


where  the  last  equality  follows  from  (2.21).  For  p’(0»B) 


l  JW  (j,dx)P(6  =0,Y*eB|6  =j,Y*=x) 
j=0  S  k  k  o  o 


=  /p(dx)P(6k=0,Y*eB|Y*=x) 


=  /  p(dx)[P(Y,eB|Y  =x)  -  p<J>(Bn( j  ,A))]  +  /  p(dx)P(Y  eB|Y  =x) 

/  •  *  \  KU  V  /  •  a  KU 


(j0.A) 


(jo’A) 


=  /v(dx)P(YkeB|Y0=x)  -  u(  jQ,A)p<t>(Bn(  jQ,A))  =  p'  (0,B), 


where  the  last  equality  follows  from  the  stationarity  of  p  for  {Y^}. 

Since  p’is  stationary  for  {S^jY*},  if  follows  from  Puri  and  Tollar  (1985) 


/A;,<1-^a.y.^>%I0  b'(i,dy)E(j,y)<1k> 


(2.25) 


(j0,A) 


that 


I 


4 

a? 


& 

I 


$ 


1 


iffij 


«s 


i 


£ 


i« 


i 


i 


$ 


id 


Since  {T*^}  is  independent  of  6Q,  we  see  that 


/  M'(l,ay)E(liy)(T*)=»<j0.MpE<,<T*). 


(2.26) 


<VA) 


Also, 


I  Jv’(j,dy)Ef  v.(T*)  =  /y'(dy)E,  (T*) 
j=o  s  v  j  »y  •>  K  g  y  K 


(2.27) 


=  /y(dy)Ey(Tk)=kEy(T1). 

s 


Since  E  (T,  )=  l  rn.E. (T. ) = 8,  combining  (2.25),  (2.26)  and  (2.27),  we 
u  i  j£j]]  1 

see  that 


E^(T*)<[py(j0,A)]“;ik8. 


Therefore,  from  (2.24)  and  the  basic  renewal  theorem  (see  Karlin 


and  Taylor  (1975))  we  have  that 


lim  P^(Y(t)eC)  =  CE^T*)]"1/  P^(Y*(t)eC,T*>t)dt. 


The  set  {y :P(N=°°lY*=y)  >  0)  must  have  ^-measure  zero,  or  else 


E  (t*)=»  by  simple  arguments  using  $-irreducibility .  Therefore 


lim  P  (Y(t)cC) = lim  P  (Y(t)eC), 
t-*»  ^  t-*°»  v 


completing  the  proof.  0 


In  the  subsequent  section,  these  results  will  be  used  to  prove  that 


a  certain  storage  model  converges  in  distribution  asymptotically. 


3.  THE  STORAGE  MODEL 

For  all  j  e  J,  let  {(U  (i),V  (i),W  (i)),  n  =  1,  2,  ...}  be  an  i.i.d. 

n  n  n 

triplet  sequence,  independent  of  {(X  ,T  )}  as  in  section  1,  and  of  all 

n  n 

{(Un(j),Vn(j),Wn( j)),  n  = 1,  2,  ...}  for  j  *  i.  Define  a  two  compartment 
storage  model  recursively  by 


(Z.  .Z_  )  =  (maxru  (X  )  +  Z_  -V  (X  ),  03, 

l)ii  2  )H  n  n  l,n-l  n  n 


(3.1 


maxCmintU  (X  )  +  Z,  ,,V  (X  )3  +  Z_  .  -H  (X  ),  0]), 
n  n  l,n-l  n  n  2,n-l  n  n 


with  the  amount  in  storage  at  time  t  being  given  by 


(Z1(t),Z2(t)) = (z1>N(t)’Z2,N(t))’ 


for  N(t )  as  in  (1.4). 

Note  that  (X  ,  Z,  ,  Z.  }  is  a  Markov  chain  on  some  subset  of 
n*  l,n*  2,n 

J  x  CO,-) x  C0,»). 

Equation  (3.1)  is  the  two  compartment  storage  model  considered  by 
Tollar  (1986),  which  has  been  widely  analyzed  in  various  forms  by  Puri, 
Balagopal,  Senturia  and  Woolford,  among  others. 

Let  us  define  E  U  by 

TT 


with  similar  definitions  for  E  V  and  E  W.  We  will  assume  E  U<®, 

v  it  it 

E^V  <»,  E^W  < ®.  Tollar  (1986)  analyzed  the  asymptotic  behavior  of 
(Z^(t),Z2(t))  for  the  various  orderings  of  E^U,  E^V  and  E^W.  However, 
the  case  where  E  U<E  V  and  E  U  <  E  W  was  left  as  an  open  question. 

7T  TT  IT  TT 

Using  the  results  of  section  2,  we  will  establish  the  main  result  of 
this  section. 


THEOREM  3.1.  If  B<“,  E  U  <  E_V  and  E_U < E_W,  then  for  arbitrary  initial 

— — —  TT  IT  -  7T  TT  - - -  1  1  * - 

distribution  (Xq^  q,Z2  q) 


lim  P(Z  (t)  <z^,Z^(t)  - z 2 ) 


09 

=  ^(T*)]"1/  P^Zf(t)lz1,Z*(t)iz2,T*>t)dt, 


for  i>,  Z*(t),Z*(t)  and  T*  as  in  section  2. 


The  majority  of  the  proof  is  devoted  to  the  non-trivial  task  of 

illustrating  there  is  a  measure  <p  for  which  (X  ,Z  ,Z  }  is  ^-irreducible, 
c  n  l,n  2,n 

and  then  establishing  ergodicity.  After  this  is  completed,  theorem  2.3 

can  be  used  to  establish  the  result. 

Observe  from  Tollar  (1986)  that  for  initial  values  (Z^  ’ 

(Z,  ,Z-  )  can  be  written  in  closed  form  as 

l,n  2  ,n 


Z.  (Z.  ) =max(Z  +S 

l,n  1,0  1,0  n 


,  max  (S  -S . )) 
*  .  n  i 

l<j<n  J 


9 


(3-2) 


»*? 

•V 


f$l 


Z2,n(Zl,0’Z2,0>  =  “xi:Zl,0  +  Z2,0  +  V  Zl,0  +  ^^VVV’ 


max  (S,  -S  .+R  -R,  )3  -  Z,  , 
.  ^ .  k  3  n  k  l,n 
1*3 ^k*n 


n  n 

where  S  =  7  (U. (X. )-V. (X. ) ) ,  and  R  =  7  (U.(X. )-W. (X. )).  Typically, 
n  .  ,  1  I  1  1  n  11  11 

i=l  1=1 


Zl,n(Zl,0)  and  Z2,n(ZlJ0lZ2,0)  wil1  be  written  simply  aS  Zl,n’  Z2,n 


with  the  Z  and  Z0  n  being  understood. 

1  )U  Z  ) u 


Using  (3.2)  and  (3.3),  the  first  step  of  the  construction  of  the 


measure  $  is  the  following  lemma. 


LEMMA  3.2.  If  E  U  <  E  V  and  E  U  <  F  W,  then  there,  exists  a  z  and  a  j 


such  that  for  every  (x_,y_),  there  is  an  nrt  with  the  property  that 


P(Xn0=j0’Zl.n0=°’Z2,n0lzl: VVZ1,0=VZ2,0= V  >  °' 


PROOF.  By  a  straightforward  alteration  of  lemma  3.1  of  Puri  and  Tollar 


(1985),  since  E  U<E  V,  E  U<E  W,  it  follows  that  there  exists  an  e>0,  an  n 

*  V  TT  IT  It 


and  a  sequence  jQ,  j^,  ...,  j^_1  such  that 


P(Sn<-e,Rn<-e,  • max  (sn-Sj)=0,Xl=:5l,‘  “  ’Vl^n-l’VVVV  >  °' 


0£j  in 


Let  M  be  an  integer  where  ^x0+^0+e^*  and 


A.  =  {u:S  .-S  .  <-e ,R  . -R  .  max  (S  .-S,  )  =  0, 

3  nD  n3~n  nj  ^3-n  nj_n<k<nj  "3  k 


Xnj-n+rjl’-*-,Xnj=V* 


*  *>  W  -  J.  •>  >  .  ■  «  »  k  •  »  *  > 


w*’r«  ■’I  .TP 


WC.WWf.V  V.V.V 


m?. 


M  M 

Then  P(  n  A. |X  =j  ) >  0.  Note  for  u e  A  =  n  A.,  that  for  n  =  nM 


j=l  3  °  °' 


j=l  3 


i)  S  <  -  Me  <  -  x  -  y  -  e , 

n0  0  o 


ii)  max  (S  -S.)  =  max  (  max  (S  ,-S.)+(S  -S  ))  =  0. 
l<jlnQ  n0  3  1*<M  nk-n*j<nk  nk  3  n0  nk 


Thus ,  for  all  ueA, 


2,  (ii)l  =  max[  max  (S  -S.),  x.+S  ]  =  0. 
1»n0  l<j<n0  n0  3  0  n0 


(3.4) 


Define  n.  by 


=  {  i :  ni  <:  j  <n( i+1 ) } . 


Then,  since  max  (S  -S  . )  =  0  for  all  k,  all  ujeA,  we  have 

°£js\  3 


(3.5) 

max  (S  -S.)=max[  max  (S  -S  . )  +  S,  -  S  ,  max  (S, -S. )]  =  max  (S  -S  ). 
0-j-k  k  3  0<j<nk  nk  3  k  nj  n^jik  k  3  k  3 


If  we  in  addition  observe  for  u>  €  A  that  <  -xQ-y0~e,  we  have 
from  (3.3)  and  (3.4)  that 


Z0  (oj)  =  max[  max  (S. -S.+R  -R.),  x.  +  max  (S, +R  -R,)]. 

2’"0  k  3  n0  *  0  Uk<n„  k  n0  ^ 


(3.6) 


Clearly,  from  (3.5) 


max 

l<j<k<nQ 


(S.-S.+R  -R,  )  <  max  (R  -R.+  max  (S.-S.)) 

3  n0  *  0<k<nQ  n0  *  Osj^k  k  3 


=  max  (R  -R.  +  max  (S, -S.)) 
Oikin0  n0  *  nk<j<k  k  3 


=  max  C  max  (R  -R.+  max  (S.-S.))] 
1££<M  nt-n£k£nt  n0  n£-n^jik  3 


^  max  [  max  (R  «-R, +S, -S, )], 
1<£^M  nl-n^<k<nl  k  k  J 


where  the  last  inequality  follows  from  R  -R  » <  -  (M-Z)e. 

n0  nZ 


Also ,  we  have  that  for  me  A, 


max  ( S.+R  -R,  )  =  max  [S  +  (S, -S  )  +  (R  -R,  )  +  (R  -R  ) 

l<ksn„  k  n0  *  l<k<n0  "k  k  "k  V"  ^  n"  "'•+n 


0  k 


Since  S  <  -  en,  n  ,  and  R  -  R  .  <  -  e(nrt-n,  -n)n  \ 

nk  k  n0  Vn  0  * 


max  (S.+R  -R  )  <  max  (S  -S  +R  ,  -R,  )  -  e(M-l). 
l<k<nQ  k  n0  *  l<k<nQ  k  nk  nk+n  * 


Since  e(K-l) >  xQ,  we  see  that 


x. +  max 
ISkSn. 


(S.+R 
k  n„ 


-R,  )  <  max  (S.  -S  +R  -R,  ) 

*  l£k£n0  k  "k  V”  ' 


< 


max  [  max  (S.-S  »  +R  »-R 
1<£<M  nl-n<k<nt  k  rU'~n 


k 


Combining  (3.7)  and  (3.8),  from  (3.6)  we  see  that  for  we  A 


i  max  (  max  (S.-S.+R  a-R,)) 
ls&SM  n£-nijikin£  k  3 


(3 


Therefore,  if  z  is  such  that 


P(S  <-e,R  <-e  max  (S  -S.)  =  0,  max  (R  -R,  +S,  -S . )  i  z, 
n  n  ....  n  ]  ’  . .  ,,y  n  k  k  ] 

Oil  in  Oiiiksn 


-VJolxo*V*<>0' 


it  follows  from  (3.4)  and  (3.9)  that 


p(xB  aj0.Z,  n 

n0  0  1*no 


=°,z2i  £zlxc 


=J0*Z1.0**0'Z2.0*x0)><M>0-  0 


THEOREM  3.3.  If  E  U  <  E  V  and  EU<E  W,  then  there  exists  a  z  and  a 

-  -  7T  7T  -  II  IT  "■  . . .  .  U 

such  that  for  every  (i,z^,z^)  there  is  an  n^  with  the  property  that 


P(Xn1=j0’Z1,n1=0-Z2,n1-zlX0=i’Zl,0=Irz2,o=I2) * °' 


PROOF.  From  (3.1)  it  is  clear  that  for  all  n,  x,  and  j,  whenever 
x1  i y1  and  x2<y2,  then 


LEMMA  3.4.  Let  {(X^,Y^),  i=l,2,...,  n}  be  a  sequence  of  numbers  satifyinr 


il  11  11 

Ex,  SO,  Iy  so,  max  (  £  X.)  =  0. 
1  1  0<;j<n'j+l 


Then  there  exists  a  kG  where 


max  (  l  X*ko)+  l  yV)-  max  (  j [  x!ko))  =  0 
0£j<k<n  j+1  1  k+1  1  0£j£n  j+1  1 


holds . 


The  proof  of  the  above  is  omitted.  It  is  easily  verified  that  for 

n  k°  k  n 

kQ  equal  to  the  integer  where  £  Y,  +  max  (  £  X. )  =  max  (  ][  X.  +  I  Y.) 

kc+l  1  0£j£ko  j+1  Osjsksn  j+1  k+1 

that  the  lemma  is  true.  Using  this  lemma,  we  establish  the  next  step  in 
<J>-irreducibility. 

THEOREM  3.5.  If  E  U  <  E  V  and  E  U  <  E  W,  then  for  z  and  j  as  in  theorem  3.3 


there  exists  an  n  ,  3,  and  a  measure  p  where  P_(t0,°»))  >  0  such  that  for 
"  X  X  "  Z  1  z 

all  A  e  B  +  ,  and  0Sx<z, 


P(X^  =j.,Z  cA,Z  =0|X=jn,Z  =0,Z_  *x)fcp  (A), 

n^  1  l,n^  2,n^  0  0  1,0  2,0  z 


PROOF .  From  lemma  3.2  it  is  clear  there  must  be  a  y  where  for 


A(i)  =  {to:S  -S  .  <  -  e,  R  .  -R  .  <  -  e,  max  (S  . -S,  )  -  0, 

ni  ni-n  ’  m  m-n  .  „  ,  ,  ni  k 

1  m-n£k<ni 


ni-nsjsks£<ni 


•  <WSk'Sj)ly’  *ni-n+l=^l . Xni=V’ 


P(A(i)IX  ,  =j_) >  0. 


9  •  •  •  y 


Let  us  define 


by 


i+k 

i-n+k 


0  <  i  <  n  -  k 
n  -  k  <  i  <  n  ’ 


and  define 


B(i,k)  =  {u»:Sn+i-Sn< 


-  e ,  P.  -  R  < 
n+1  n 


e,  max  (R  -R.+S ,-S  . )  <  2y » 
i£j<k<£<nti  3 


Xitl 


n+2 


(3.12) 


Then  from  (3.11)  it  follows  that  for  all  i,  k. 


P(B(i,k)|X.=i[)k))  >  0. 


-1 


Clearly,  for  A=  n  A(i),  where  M  >  e  (z+3y)  +  l,  we  have  P(A  |  XQ= j  Q)  >  0 , 


i=l 


and  for  all  weA,  S„  <  -  Me ,  R  <  -  Me ,  max  (S  -S.)  =  0.  Thus,  from 

Mn  Mn  OSjSMn  Mn  1 

lemma  3.4  we  have  there  is  a  k^  where 


P(A,R,,  -R,  +  max  (S.  -S.)=  max  (R„ -F.+S.-S. )  |  X=jn)  >  0 .  (3.13) 

k0  0<j<kQ  k0  3  0Sj<kSMn  ^  k  k  3  °° 

Let  m  =  kQ  -  n  •  max{i:ni  <:kQ}  and  n^  =  Mn+m.  It  is  clear  from  (3.12)  and 
(3.13)  that  for 


E  =  {  n  n(ni+m,m)}  n  (oj:  max  (R  -R,  +S,  -S . )-  max  (S  -S  ,  )  =  0}, 
i=0  mSjlk^n^  nl  3  msjsn^  1 


P(B I X  =  j  ) >  0. 
m  m 

24 


(3.14) 


Also,  for 


t 


,* 

V 

$ 

i 


c={“="“w  <Rm-VS>:-Sj,S!'>  W’ 


P(CnB)X0=j0)  >  0. 

Let  us  show  for  Z^  Q=0,  Z2  Q=z  for  all  w «  C  n  B,  Z2 =0.  It 
follows  from  (3.3)  that 


Z0  =max(  max 
J,nl 


-P1e+Slf-S4>»  Z+Rn  >  “  (Sn  "V' 

Bl  ni  1  1  3  nl  0fijsn1  nl  3 


From  the  definition  of  B  and  C,  we  see 


z  +  R  =  z  +  R  +  R  -R<z  +  y-Me<0, 

n!  »  ni-  m 


and  therefore 


_  =  maxC  max  (R  -R.+S.-S. ),  max  (R_  -R.+S.-S . ) 3-  max  (S  -S. ). 

2*nl  tfjfidn,  nl  K  k  3  Oijsksn,  "l  *  k  1  OijSn,  nl  ] 

1  j<m  A 


From  (3.14),  we  have  that 


max  (R  -R, +S, -S .)  -  max  (S  -S.)<:0, 
nsjsksn,  n!  K  K  3  osjsn,  nx  3 


OSjSn, 


and  therefore  to  show  Z0  =0  for  we  3nC,  it  is  sufficient  that  for 

i  ,n^ 

C  s  j  <  m,  0  S  j  S  k  S  n. ,  R  -R.  +  S.  -  S .  i  0 .  If  k  <  m,  from  the  definition 

1  n^  k  k  j 

of  B  and  C, 
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IMUNJUUUUtAl 


From  the  previous  theorems,  we  can  now  establish  that  (X  ,  Z,  ,  Z„ 

n  l,n  2,n 

is  (fr-irreducible. 


THEOREM  3.6.  If  E^U  <  E^V  and  E^U  <EtW,  then  (X^,  n»  Z^  n)  is  $-irreduc 
ible  with  respect  to  the  measure  <f)  defined  by 

O 

♦  (B)=n2-"PC(Xn,Z1>ii,Z2in,£B|Xo=ji,Zijo=X.Z2io=0)MI<dx,> 


for  j1  and  as  in  theorem  3.5. 


PROOF .  From  definition  1  of  <(-irreducibility  (see  section  2),  we  see  we 
need  for  all  B  where  (j>(B)>0,  that  for  all  i,  z^,  z2» 


09 

U-"  P((xn,zlin,22in) «  >  0. 


If  4> ( B  >  >  0,  there  is  an  n^  where 


^(<Xn1’Zl,ni’Z2,ni)tBlX0^1’Zl,0=!<-Z2.0'0,l>z<dlI>>0' 


From  theorem  3.5  there  is  an  n2  where  for  x£z. 


P(X  =j  ,Z  eA,Z  =0|X  =j  ,Z.  =0,Z.  =x)>u  (A). 

n2  i  l,n2  2 ,n2  0  ■'0  1,0  2,0  z 


From  theorem  3.3  there  is  an  n^  where 


P(x_  =^.z;  =0,Z„  £z|X  =i,Z,  =z, ,Z„  =zrt)  =  8>0. 


l.»»  MTi 

.HSW 


Therefore,  for  n  =  +  n ^  we  find  from  theorem  3.3  and  3.5  that 


P((VZl.»-Z2.n>tB'V1-2l,0-VZ2,0**2) 


>  p(  ( Y  7  7 

VV  1’”l«s’  2-Vn2)£B,X0=10’Zl,0=0'Z2,0Si!)6 


S/P((\’Zl,n1-Z2.ni)eBIX0^1-Zl,0=,,-Z2>0=0)l‘2(d’<)6>0-  D 


Now  that  <J)-irreducibility  has  been  established,  the  ergodicity  of 


{X  ,  Z.  ,Z.  }  can  be  established  by  appealing  to  the  large  body  of 

n  i  *n  z  ju 


literature  on  (^-irreducible  Markov  chains. 


THEOREM 


3.7.  If  E^U^V  and  EffU  <  E^W ,  then  {(X^Zj.  n>Z2  n»  is  ergodic. 


PROOF.  Since  {X  ,  Z,  ,  Z„  }  is  ^-irreducible ,  from  Jain  and  Jamison 
-  n  l,n  2,n 


(1967)  it  follows  that  there  exists  a  sub invariant  measure  y  where 


y  >>  <!>.  (see  definition  2  in  section  2).  From  Tweedie  (1975),  it  follows 


that  if 


iiB  VlZ2,i>Cfl  VVZ1,0*0-Z2>(T0)>  >  0 


(3.15) 


for  some  A  where  0  <y(A)  <»,  then  {X  ,  Z,  ,  Z„  }  is  ergodic. 

n  i,n  2,n 


It  was  established  in  Puri  and  Tollar  (1985)  that  there  is  a  jQ,  NQ,  M 


and  6  >  0  such  that  for  all  n  >  N^, 


where  B  =  {j0>u{l,  2,  ...  M}.  Also,  from  Tollar  (1985a),  we  have  that 
„  d 

Zj  n — *■  Z2.  As  such,  there  is  a  w  where  for  all  n>NQ, 

P<Z2.n>"'X0=3o*Z1.0°0’Z2,0*0,<{/J- 
Thus ,  for  all  n  >  Nq  , 

P(VB>Z1,„=0-Z2,n*  VVZ1,0*°>Z2,0=0>  >  S/2' 

As  such,  from  (3.15)  we  need  only  establish  that  0 < y(B,0,CO,w]) < ®  to 

prove  ergodicity.  From  theorem  3.3,  it  is  clear  for  some  w  that 

$(B,0,[0,w])  >  0,  which  implies  y(B,0,[0,w3) > 0,  since 

To  show  that  y(B,0,[0,w3) <®,  it  is  sufficient  that  p(i,0,C0,w3) < ® 

for  all  i«B.  Since  u  is  o-finite,  there  are  sets  where  y(S  )<»,  and 

n 

CD 

u  Sr  =  J * [0,®) x  Co,®).  In  the  construction  of  these  sets  in  the  proofs 
n=l  n 

that  u  is  a-finite  (see  Jain  and  Jamison  (1967),  and  Orey  (1971)),  we 
see  that  the  sets  are  of  the  form 


S 

n 


n  . 

{(j,x.,x_):  £  P((X.,Z  .,Z-  .)€A|X_=j,Z,  =x.  ,Z  =x_)  <t  — ) 

^  "  XfU  x*  a  n 


(3.16) 


for  A  a  specified  set  where  $(A) >  0. 


Since  <p(A) >  0,  there  is  an  n^  where 


/P<(X„, -Zl,n, -Z2,n, 3eA 1 X0=31  •Zl,0=x’Z2,<r0)’'z<dx) * ° >  °'  (3'17) 

Sill 


From  theorem  3.5,  there  is  an  where  for  x  s z ,  all  D  e  B  , 


P(Xn2=3l>Zl.n2£D-Z2.n2=OIVio’Zl,Ol0>Z2,0=1')2',,(D)-  (3'18) 


Also,  from  theorem  3.3  and  (3.1),  it  is  apparent  that  there  is  an  n^  where 
for  all  uiw, 


m 

\% 

\m 


P(Xn1=30-Zl.n.=0-Z2.njSillV3'Zl,0=0-Z2,0=“) 


JP(Xn.=3o-Zl.nJ=0-Z2.njS2lX0=3'Z1.0=0’Z2.0=”’“Sj>0- 


(3.19) 


Therefore  from  (3.17),  (3.18)  and  (3.19)  for  nsn^+n^  +  n^*  it  is  clear 
that  for  all  uSw 


.Vi' 


P((Xn;Zl.n’Z2,n)£fllX0=3'Z1.0=0-Z2,0=“)s“Sj>0' 


It  is  therefore  clear  from  (3.16)  that  M  ,0,[0,w])  c  S  ,  where  n*>(a6.) 

J  ]  ] 

As  such  p(B,0,[0,w])  <  l  u(S *)<«,  which  completes  the  proof.  □ 

jeB  nj 

Once  the  $-irreducibility  and  ergodicity  of  (X^,  Z^  Z^  is 
established,  the  proof  of  theorem  3.1  follows  quickly. 


I 


:Y .V 70  ’V:,t*VvV’V 


Mmm® 


PROOF  OF  THEOREM  9.1.  The  majority  of  the  proof  is  accomplished  by  simply 

noting  that  {X  ,  Z.  ,  Z„  ,  T  }  has  the  desired  form  of  section  2.  As 
n  l  ,n  2  »n  n 

such,  theorem  2.3  can  be  applied  to  yield  for  <j>-almost  all  (i0,xQ,y0), 


^(i  x  v  (t)^2. jZ-CtJSz^) 

lx0,x0*y07  1 


=  CVTN)]”1{  p^(Z?<t)S2l,Z*(t)Sz2,T*>t)dt. 


(3.20) 


To  complete  the  theorem,  it  is  sufficient  to  show  that  for  all  (ip,x0,yQ), 
(3.20)  holds. 

Since  (3.20)  holds  for  ^-almost  all  initial  values,  for  any  iQ e  J, 
there  is  an  (x^y^)  where  for  initial  value  (i^x^.y^),  (3.20)  holds. 

As  such,  to  complete  the  proof  it  is  sufficient  to  show  that 


P[(Zl,n(xO),Z2,n(xO’yO))  *  ^l.n**!1  ,Z2,n(xl’yl))  i,O’|X0  =  10]  =  °* 


for  Z,  (x),  Z-  (x,y)  as  in  (3.2)  and  (3.3). 
x,n  ^  ,n 

From  (3.2)  and  (3.3)  we  can  see  that 


’>t(Zl,n<X0),Z2,n(  W’  *  (Zl,n(xl)'Z2,n(xl’yl))  i'°-  IX0  *  V 


SP([S  >min(-x-,-x, )] u [  max  (S.+R  -R. )>-min(-x. ,-x. )] 
n  01  l<j<n  1  n  3  0  1 


(3.21) 


u  [Rn>min(-x()-y0,-x1-y1]  i.o-  |x0=i0>- 


31 


From  Chung  (1967), 


-t 

n  X  +  EO-EV  a.s . ,  n~AR  +EU-EH  a.s., 
n  ir  ir  n  it  7t 

an<*  n  ^  max  (S  ,+R  -R. )  =  n  +n  max  (S  .-R.) 

lsjsn  3  n  j  n  isjsn  3  3 

+  E  U  -  E  H  +  max(  0  >E  H-E  V) ,  a.s.. 
n  it  it  it 


Since  E^U < E^V  and  EffU<EirW,  we  see  that 


S 

n 


-►  —co 


9 


R 

n 


max  (S.+R  -R. )-►-»,  a.s., 
lsjsn  3  n  3 


and  therefore  from  (3.21)  we  have  that 


PC<Zl,n<1!0)-22,n(Vy0))‘(Zl.n(xl)’Z2,n<xl>5rl))1-o-IX0=i0)s0- 


which  completes  the  theorem.  □ 

As  is  usually  the  case,  the  construction  of  the  measure  4>  was  the 
major  difficulty  in  dealing  with  the  storage  model  as  a  Markov  chain. 

The  technical  details  unfortunately  obscure  the  simplicity  of  the  con¬ 
cept.  When  there  is  a  renewal  point,  the  measure  $  is  readily  constructed 
for  general  Markov  chains.  While  there  is  no  renewal  point  in  the  pre¬ 
sent  model  we  make  the  process  act  like  one  exists  by  first  visiting 
(iQ,0,*)  and  then  visiting  (i  ,»,0).  In  this  manner  it  "forgets"  the 

initial  values  Z,  „,Z„ 

1,0*  2,0 


4. 


CONCLUSION 


While  the  results  in  section  2  are  useful  for  the  typical  storage 
models  defined  on  denumerable  state  Markov  renewal  processes,  they  are 
not  particularly  satisfying  for  the  more  general  Markov  renewal  process 
on  an  arbitrary  state  space.  Perhaps  more  structure  on  {T^}  (for  example, 
absolute  continuity  on  all  sojourn  times  with  respect  to  a  single  measure) 
would  allow  a  suitable  modification  of  the  C-set  proof  to  establish  that 
ergodicity  and  an  appropriate  finite  sojourn  moment  are  sufficient  to 
satisfy  the  hypothesis  of  section  2.  If  this  were  the  case,  the  proof 
that  the  arbitrary  semi-Markov  process  converges  would  be  complete. 

This  area  remains  an  open  question. 

The  multitude  of  steps  in  section  3  point  out  a  recurring  problem 
in  Markov  chains,  the  construction  of  $.  The  techniques  of  section  3 
shed  little  light  on  how  to  construct  such  measures.  As  of  new,  the 
technique  is  very  model  specific.  For  applications,  it  would  be  very 
useful  to  have  conditions  which  guarantee  <(>-irreducibility  for  Markov 
chains. 

As  far  as  the  actual  model  under  consideration  in  section  3,  there 
are  a  variety  of  areas  for  further  research.  The  most  obvious  is  to 
take  the  arbitrary  compartment  model  in  Tollar  (1985a, b),  and  extend  the 
continuous  time  results  as  in  the  present  paper  and  Tollar  (1986). 

Perhaps  of  more  interest  would  be  to  alter  the  model  to  more  realistically 
accomodate  two-way  flow  between  compartments.  Finally,  it  would  be  nice 
to  have  a  more  useful  characterization  of  the  limiting  distribution  of 
(Z^(t),Zj(t))  than  the  renewal  equation  results  found  in  section  3. 
Unfortunately,  the  present  techniques  seem  to  be  of  little  help  in  these 


directions. 
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